Distracted driving detector

ABSTRACT

Methods and apparatus for detecting a distracted user condition. In embodiments, a handheld device, such as a mobile phone, includes a distraction detection module to process sensor data and/or user date to generate a score indicative of whether the user is distracted, such as texting and driving. In embodiments, a keyboard of the device can be disabled upon detection of a distracted user condition.

BACKGROUND

As is known in the art, distracted drivers can cause dangerous andhazardous conditions. For example, using a handheld keyboard, such astexting, while driving makes accidents significantly more likely tohappen.

SUMMARY

Embodiments of the invention provide methods and apparatus for enablinghandheld devices, such as mobile phones, tablets, and the like, todetect if a person is driving and interacting with the handheld deviceat the same time. Embodiments should have relatively low false positiverates to minimize user frustration due to incorrect detections, such astexting by a passenger. In some embodiments, a handheld device can warnthe user of distracted driving detections, such as with beeps and/orwarning messages on the screen, or disable the handheld device controls,e.g., disable the keyboard.

In embodiments, a handheld device includes a detection module configuredto process information from a plurality of device sensors and/orhistorical user information to detect a distracted driving condition.The device may differentiate a driver from passengers so that passengerswill not be impacted by false positive detections. A front camera of thehandheld device can monitor user eye and head movements to determine ifeyes are alternating between the screen and the road, for example.Information from a variety of sensors can be processed and weighted todetermine whether a distracted driving condition exists.

In one aspect, a method comprises: receiving sensor informationincluding GPS data to determine whether a device under control of a useris moving relative to Earth surface; receiving sensor informationincluding vibration levels of the device; receiving sensor informationincluding angle orientation information for the device; receiving sensorinformation including first camera information for the device to detectuser head and eye movement; processing the sensor information todetermine a score corresponding to a likelihood of a distracted usercondition; and communicating with a keyboard module of the device tomodify at least one setting for operation of a keyboard controlled bythe keyboard module.

A method can include one or more of the following features: receivingsensor information including data from a first camera of the device andprocessing eye movement of the user, receiving sensor informationincluding touch and type information for a user typing on the keyboardof the device, processing the touch and type information to determinewhether the user is one-hand typing or two-hand typing on the keyboard,processing the touch and type information for error rate comparison,processing the touch and type information for finger surface area on thekeyboard, processing the touch and type information for speed of typingcomparison, processing historical driving information for the userincluding time of day historical driving information, receiving sensorinformation including local wireless connection information, receivingsensor information including second camera information from the devicethat includes light level, receiving sensor information including datafrom a proximity sensor of the device, receiving sensor informationincluding data from a light sensor of the device, receiving sensorinformation including data for a number of other nearby devices,receiving sensor information including acoustic information detected bythe device to determine a number of persons in the vehicle, receivingsensor information including face and behavior information of the user,generating a signal for an audible alert corresponding to the scorecorresponding to a likelihood of a distracted user condition being abovea selected threshold, and/or modifying at least one setting foroperation of a keyboard controlled by the keyboard module includingdisabling the keyboard.

In another aspect, an article comprises: a non-transitorycomputer-readable medium having stored instructions that enable amachine to perform: receiving sensor information including GPS data todetermine whether a device under control of a user is moving relative toEarth surface; receiving sensor information including vibration levelsof the device; receiving sensor information including angle orientationinformation for the device; receiving sensor information including firstcamera information for the device to detect user head and eye movement;processing the sensor information to determine a score corresponding toa likelihood of a distracted user condition; and communicating with akeyboard module of the device to modify at least one setting foroperation of a keyboard controlled by the keyboard module.

An article can further include instructions for one or more of thefollowing features: receiving sensor information including data from afirst camera of the device and processing eye movement of the user,receiving sensor information including touch and type information for auser typing on the keyboard of the device, processing the touch and typeinformation to determine whether the user is one-hand typing or two-handtyping on the keyboard, processing the touch and type information forerror rate comparison, processing the touch and type information forfinger surface area on the keyboard, processing the touch and typeinformation for speed of typing comparison, processing historicaldriving information for the user including time of day historicaldriving information, receiving sensor information including localwireless connection information, receiving sensor information includingsecond camera information from the device that includes light level,receiving sensor information including data from a proximity sensor ofthe device, receiving sensor information including data from a lightsensor of the device, receiving sensor information including data for anumber of other nearby devices, receiving sensor information includingacoustic information detected by the device to determine a number ofpersons in the vehicle, receiving sensor information including face andbehavior information of the user, generating a signal for an audiblealert corresponding to the score corresponding to a likelihood of adistracted user condition being above a selected threshold, and/ormodifying at least one setting for operation of a keyboard controlled bythe keyboard module including disabling the keyboard.

In a further aspect, a device comprises: a processor and a memory; adistraction detection module to receive sensor information including GPSdata to determine whether a device under control of a user is movingrelative to Earth surface, wherein the distraction detection module isconfigured by the processor and the memory to receive sensor informationincluding vibration levels of the device from a gyro module, to receivesensor information including angle orientation information for thedevice, to receive sensor information including first camera informationfor the device to detect user head and eye movement, to process thesensor information to determine a score corresponding to a likelihood ofa distracted user condition; and a keyboard module coupled to thedistraction detection module for receiving the score corresponding to alikelihood of a distracted user condition, wherein the keyboard moduleis configured to modify at least one setting for operation of a keyboardcontrolled by the keyboard module.

A device can further include one or more of the following features:receiving sensor information including data from a first camera of thedevice and processing eye movement of the user, receiving sensorinformation including touch and type information for a user typing onthe keyboard of the device, processing the touch and type information todetermine whether the user is one-hand typing or two-hand typing on thekeyboard, processing the touch and type information for error ratecomparison, processing the touch and type information for finger surfacearea on the keyboard, processing the touch and type information forspeed of typing comparison, processing historical driving informationfor the user including time of day historical driving information,receiving sensor information including local wireless connectioninformation, receiving sensor information including second camerainformation from the device that includes light level, receiving sensorinformation including data from a proximity sensor of the device,receiving sensor information including data from a light sensor of thedevice, receiving sensor information including data for a number ofother nearby devices, receiving sensor information including acousticinformation detected by the device to determine a number of persons inthe vehicle, receiving sensor information including face and behaviorinformation of the user, generating a signal for an audible alertcorresponding to the score corresponding to a likelihood of a distracteduser condition being above a selected threshold, and/or modifying atleast one setting for operation of a keyboard controlled by the keyboardmodule including disabling the keyboard.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing features of this invention, as well as the inventionitself, may be more fully understood from the following description ofthe drawings in which:

FIG. 1A is a front view of a handheld device having distractiondetection and FIG. 1B is a back view of the device of FIG. 1A;

FIG. 2A is a representation of a handheld device in a vehicle while notbeing used by a driver and FIG. 2B is a representation of a handhelddevice in a vehicle with the driver using the device keyboard, and FIG.2C shows an example device angle position;

FIG. 3 is a flow diagram of an example sequence of steps for determininga distracted situation;

FIG. 4 is a flow diagram showing an example sequence of steps forprocessing sensor and other information to make a distraction detectiondetermination;

FIGS. 5A-5D are a tabular representation of example sensor data withexample weighting; and

FIG. 6 is a schematic representation of an example computer that canperform at least a portion of the processing described herein.

DETAILED DESCRIPTION

FIG. 1A (front view) and FIG. 1B (back view) show an example device 100having sensors, user interface controls, and components that enableprocessing of sensor data and/or user data to determine whether adistracted driving condition exists, such as texting and driving. In anillustrative embodiment, the device 100 is provided as a handhelddevice, such as a mobile phone.

In embodiments, the device 100 includes a display 102, such as a touchscreen, user interface buttons 104, one or more speakers 106 and amicrophone 108. The device 100 can include a front camera 110 and a backcamera 112. Without limiting embodiments of the invention to anyparticular configuration, it is understood that front and back arerelative terms and that the front camera 110 can be considered thecamera that faces the user in normal use.

The device 100 can include a light sensor 114 and a proximity sensor 116each of which can be located in any practical position on the device.Information from the light sensor 114 and proximity sensor 116 aredescribed more fully below.

The device 100 includes a processor 120 coupled to memory 122 both ofwhich are supported by an operating system 124. In embodiments, thedevice 100 includes a distraction detection module 126 coupled to theprocessor 120 and the memory 122. A keyboard module 128 is coupled tothe distraction detection module 126, as well as the processor 120. Thedistraction detection module 126 can detect a distracted user condition,such as texting and driving, and communicate with the keyboard module128 to modify or disable device keyboard functionality, as describedmore fully below.

The device 100 can include a gyro sensor module 130 and a GPS module132. In embodiment, the device 100 includes a close proximity wirelesscommunication technology, e.g., BLUETOOTH, module 134, a wirelessnetwork communication, e.g., Wi-Fi, module 136, and a mobilecommunication module 138.

FIG. 2A shows a handheld device 100 and the user 150, shown as thedriver, during safe operation of a vehicle. In the illustratedembodiment, the device 100 is located in a cupholder 154 in the consolearea of the vehicle. In general, sensor data will be indicative of anon-distracted driving operation, as described more fully below.

FIG. 2B shows the handheld device 100 in the vehicle 152 when the driver150 is texting and driving. In embodiments, the gyro sensor 130 canprovide angle information, e.g., the angle of the handheld device whenuser is safely operating the device, such as the device being in avehicle cupholder or texting on a couch, and unsafely operating thedevice, such as texting and driving. The gyro sensor 130 can alsoprovide vibration information to detect when the handheld device 100 isstationary/idle, e.g., in a vehicle cupholder 154, and when the device100 is being actively used by the user.

FIG. 2C shows an example frame of reference and angle information for adevice 100 having, x, y, z axes based on the figure. The way x, y, z iscalculated could be different depending on the device. In theillustrated position, gyro sensor 130 outputs an angle position in x,y,zcoordinates as [55, −15, 30] where reference position is [0, 0, 0] whenthe phone is flat and oriented in a given position. It is understoodthat any suitable reference frame and coordinate type can be used tomeet the needs of a particular application.

Referring again to FIGS. 1A and 1B, in embodiments, the gyro sensor 130detects angle of the device 100 in the vehicle including when the driveris holding the device. The angle and acceleration of the mobile device100 can suggest a positive case indicative of texting and driving. Thatis, the angle and peak and average acceleration of a device 100 duringtexting and driving is usually different than non-texting and drivingconditions of the same user in order to accommodate steering wheelposition and multitasking needed to continue driving at the same time,as shown in FIGS. 2A and 2B. Also, angle of the phone 100 and peak andaverage vibration will likely be also different between times when theuser is driving the car and when in the passenger seat or texting on acouch.

Gyro-accelerometer sensor 130 data can also be used to examine movingand typing patterns. Most drivers who text and drive start texting whenthe car stops at a traffic light, and they stop typing when car startsmoving again. Gyro-accelerometer 130 data and touch and type data can beused together to indicate driving situations.

Another sensor data that may be utilized for detecting unsafe operationis the vibrations detected by the gyro sensor (gyroscope-acceleratorcombo sensor) 130. For example, many text-and-drivers keep their phone100 in the vehicle cup holder 154 or other location when not using thedevice. When in such a location, the phone 100 is typically subject tomore vibration and impacts due to road conditions. Passengers are lesslikely to place a device or phone in a cup holder or similar location.

In one scenario, a device is idle and the screen is locked. Uponunlocking of the screen by the user, the distraction detection module126 can collect sensor information. For example, the distractiondetection module 126 can examine sensor information to determine whetherthe device is in a pocket or a bag. If so, then the distractiondetection module 126 will generate a score indicative of a non-textingand driving situation. For example, in a user's pocket, the device canbe at any angle but will be subject to low vibration levels becausethere are several shock absorbers for the device, such as the seat,user's body, clothes, etc. The sensor data for the proximity sensor,light sensor, cameras, etc., will also be indicative of being in apocket or bag. Where the device 100 is in a cup holder, the device canbe at angle but will likely experience relatively high vibration levels.However, if the device is flat on its front or back surface, it isunlikely the device is in a pocket.

In another example, the device 100 may be held by a user with the screenunlocked. For a driver, vibration may not be a heavily weighted factorwhile the device angle may be weighted heavily along with how frequentlythe angle changes and with what acceleration. In general, most drivershold the phone differently than when they are not driving and theychange angle as they stop and go, or when they see a law enforcementperson, for example. For a passenger, vibration levels may not be ofparticular interest while the angle of the device may be of interest.

The front camera 110 can detect movements of user head and eyes. Inembodiments, the eyes and head of the user may alternate between thescreen of the mobile device 100 and straight ahead towards thewindshield (FIGS. 2A and 2B). If the user view alternates between thedevice 100 and the windshield above a threshold amount, e.g., more than3 switches in a 5 second window, a texting and driving situation may beindicated.

The front camera 110 and back camera 112 can detect the light level inlumens. For example, the light levels between (a) the user outside of acar, (b) in a passenger seat, and (c) in the driver seat will bedifferent due to physical characteristics of the environment, sincetypically less light exists in the driver seat due to steering wheelthan front passenger seat. The same approach applies to camera focusdata calculating distance, as the driver's handset device will have ashorter distance to the next object because the steering wheel or mainconsole will be very close by. Light level and focus data may be used toindicate driving situation. It is understood that light levels detectedby the device cameras 110 and 112 as well as the light sensor 114 may beused for the same purpose.

The proximity sensor 116 detects if an object is close to the device andcan be useful for detecting two hand typing, as described more fullybelow. It is understood that single hand typing may be indicative of atext and driving situation, although some users may be able to drive andtype with two hands. The sensor data source to detect two hand typingincludes keyboard touch and type and proximity sensor complements it tofurther reduce false positives and false negatives.

In embodiments, wireless communication information can be used todetermine whether a distracted driving condition may exist. For example,based on the available networks, such as BLUETOOTH connections, and thenames of the networks, the wireless communication module 134 candetermine whether a user is a driver or in a public location, such as abus or other public transportation. If a relatively high number ofBLUETOOTH connections are detected, this may be indicative of anon-driving situation. In addition, network names may be suggestive of apersonal vehicle and may be indicative of a user being a driver. Inaddition, the number of times a user has connected to a given networkmay also be taken into account by the wireless network module 136 indetermining whether a distracted driving condition exists. Inembodiments, a device protocol, such as for IPHONE or ANDROID systems,may be used by the mobile communication module 138 to determine that theuser is in a public transportation environment where there are manynearby phones.

Statistical and historical data can also be used by the distractiondetection module 126 to determine whether a distracted driving situationmay exist. For example, time of the day information shows that thedriver usually drives 9-10 AM and 6-7 PM on weekdays. Based on thecurrent time and date, the driver is likely in the vehicle which can beused as a factor in determining whether a distracted driving conditionexists.

Keyboard touch and type information, using a device 100 touch screen 102and keyboard application, can also be used to determine if a distracteddriving condition exists. For example, if typing on the keyboard is asingle hand operation then this may be indicative of a distracteddriving condition since most users can only drive and text with onehand. It should be noted that some users can drive and use both hands onthe keyboard to type at the same time. One hand typing versus two handtyping can be detected by observing the speed of touching the keys. Twohand typing will likely touch non-adjacent keys significantly fasterthan one hand as there will be extra delay caused by thumb moving fromone key to another. The gyro-accelerometer sensor 130 can also act as asupplemental data to detect if the device is held on portrait orlandscape position. Landscape position very likely means this is atwo-hand typing situation, hence it is less likely a driving situation,although some drivers can text and drive using two hands.

In addition, a number of typing errors and deletion rate can be comparedto averages for a given user. A relatively high error rate can beindicative of texting while driving. Such processing can be performed bythe distraction detection module 126.

In embodiments, a surface area of the touch of the fingers can becompared to averages for a given user. In a driving situation, due tomultitasking, user fingers will typically touch a larger surface area onthe touch screen 102 than non-driving situation.

In embodiments, the typing speed and/or the time it takes for each touchof the keyboard can be taken into account. For example, significantlyslower than average type speed can be indicative of a distracted drivingcondition. Additionally, the touch time, which is the time between afinger touched the screen and lifted, will likely be more for thedriving situation. In embodiments, such processing can be performed bythe distraction detection module 126.

In some embodiments, a car sensor, such as a seat pressure sensor, candetermine whether a driver and/or passenger is present in the vehicle.This information can be used to determine whether the user is a driverand alone in the car. For example, if the car sensor tells the handsetdevice there is no passengers in the car except the driver, then theuser of the handset device is very likely the driver.

In embodiments, acoustic information from the vehicle or devicemicrophone can be processed to determine whether conversation are takingplace. If there is conversation between two or more people in thevehicle, it will imply the user of the handset device could be a driveror passenger, however, if there is no conversation detected then it isvery likely there is only one person in this car who is the driver, soit is a text and drive situation. The acoustic information will be morereliable if it can use voice biometrics to differentiate a realconversation happening in the vehicle from a conversation on the radioor a monologue.

In embodiments, the front camera 110 can analyze a user's face and/orbehavior, age, gender, mood, and other face attributes in determiningwhether a distracted driving condition exists. For example, certain ageand gender groups could have statistically higher likelihood of text anddriving. Additionally, user's face attributes and mood may be used tofurther understand whether this is a driving condition. For example, ifthe user's eye brows look significantly different than in the sameuser's historical safe texting, i.e. raised eye brows, this may indicatetexting and driving.

In an example scenario a user is driving a car and typing with thekeyboard of a device 100, such as a smartphone. Typing on the keyboardcan be intended for text messaging, entering a web address into thebrowser, etc. The distraction detector module 126 receives location data(GPS, Wi-Fi, cellphone tower triangulation) for the device anddetermines that the device is moving faster than a speed threshold, suchas 20 miles per hour, which indicates that the device is in a movingvehicle. At this point, it is understood that a driver or a passengercan be using the device.

The distraction detector 126 collects sensor and/or historicalinformation for the user and generates a score indicating whether it islikely the user of the device is in a distracted driving situation ornot.

While example embodiments of the invention are shown and described inconjunction with texting and driving it is understood that embodimentsof the invention are applicable to detecting distracted situations ingeneral, such as walking and texting, for example, in a city with heavytraffic and many objects. In such applications, sensor informationbaseline information can be adjusted for detecting walking and texting.

FIG. 3, in conjunction with FIGS. 1A and 1B, show an example sequence ofsteps for determining a distracted user condition and communicating witha keyboard application of a handheld device. In step 300, a user wantsto type on the keyboard of the handheld device so that the keyboardmodule 128 is initiated to interface with the user, for example bytouchscreen. In step 302, the keyboard module communicates with thedistraction detection module 126 to ascertain whether the user isdriving. In step 304, the distraction detection module 126 receivesinformation from the gyro sensor module 130 including deviceacceleration data. In step 306, it is determined, such as by thedistraction detection module 126, whether the speed and location, suchas from a GPS module 132, indicate that the user is in a moving vehicle.The speed and location data can correspond to a desired time interval,such as the last five minutes. If not, in step 308, the distractiondetection module 126 communicates with the keyboard module 128indicating that a distracted driving condition does not exist, e.g., adriving and texting situation is not present. In optional step 310, thedistraction detection module 126 can obtain sensor data to build orupdate a safe condition baseline. For example, sensor data for thevarious sensors, such as gyro, front and back cameras, proximity sensor,touch and type, light sensor, wireless communication, wireless network,and sound information can be updated for a safe driving condition. Instep 312, the keyboard module 128 can allow the user to type on thekeyboard and otherwise interface with the device.

If the user was found in step 306 to be in a moving vehicle, in step 314the distraction detection module 126 receives sensor data and/or userdata to determine a score indicative of the likelihood of a distracteduser situation, such as texting and driving, in step 316. In step 318 itis determined whether the score is above a threshold. In an embodiment,a score above the threshold indicates that a determination of distracteddriving is present. If the score is below the threshold, the distractiondetection module 126 communicates to the keyboard module 128 that adistracted driver situation is not present. In step 322, the keyboardmodule 128 can allow the user to type on the keyboard.

If the score in step 318 was determined to be above the threshold, instep 324 the distraction detection module 126 can communicate to thekeyboard module that a distracted driver condition is present. Inembodiments, the score computed in step 316 can be provided to thekeyboard module. In step 326, the keyboard module 128 can take actionsin response to the distracted driver condition. Example actions includegenerate a warning to the user, log the sensor and/or other information,and/or disable the keyboard, etc.

In step 328, from either step 324 (text and drive situation) or step 320(non text and drive situation), the system can perform machine learningto improve the detection of distracted user conditions with increasedaccuracy and decreased false positives.

In embodiments, a distraction detector can include machine learning. Thedevice is initialized with a set of standard thresholds for the sensors(e.g., front and back camera, proximity sensor, statistical information,touch and type, light sensor, wireless connections, sound information,being monitored to detect the driving mode.

The initial information baseline is established by generating modelsfrom collecting data of users in multiple control groups. In oneembodiment, control groups include a first group of users in thepassenger seat of moving cars and a second group of users on a videodriving and texting. From collected data, baselines are established thatcategorizes appropriate thresholds for the initial settings. Inembodiments, initial settings are downloaded to a handheld device uponinitiation of the distraction detector.

In embodiments, when the driver attempts to perform a texting operationand the system compares the settings with thresholds on the device, thesystem locally stores the settings and the determination of distracteddriving. The settings contain a snapshot of the above settings and thedetermination of distracted driving along with information regardinguser overrides. The device, for example when connected via a wirelessnetwork, can upload sensor and stored data to a network for furtherprocessing.

Upon receiving the uploaded data, a network application can generatethreshold models for different hierarchical layers, such as global,regional, user, etc. The collected data is then compared with variousthe models and the model thresholds are improved as patterns emerge. Ifthere is sufficient user-specific data, a user-specific model can bedelivered to the device. If there is insufficient data to extract morerefined thresholds for the user, regional thresholds can be delivered tothe device with thresholds based upon regional driving patterns. Globalsettings can also be downloaded in the absence of more specific models.

In some embodiments, one or more cameras with a field of view includingthe driver's face can be used for eye and head movement tracking.

FIG. 4 shows an example sequence of steps for processing sensorinformation and user information for determining a distracted usercondition. In step 400, a distraction detector retrieves a distractionscore calculation. In step 402, the distraction detector obtains sensorinformation, such as some or all of the sensor data described above. Instep 404, the current sensor data is compared with statistical datarelating to whether a distracted driving condition exists or not. It isunderstood that data can be stored locally on the device, a remotelocation, or in a cloud-based service. As shown in 406, first data 408can include data associated with safe device operation for a currentuser. Second data 410 can include data associated with unsafe deviceoperation for a current user. Third data 412 can include data associatedsafe device operation for a user baseline, such as baseline data for aset of users. Fourth data 414 can include data associated with unsafedevice operation for a user baseline, such as a set of users.

In step 416, the sensor data can be normalized with a desired weightingscheme to generate a score indicating whether or not a distracted drivercondition exists based on the sensor and other data. In exampleembodiments, in step 418, the distractor detector can process the datato generate the score. A flag can be set to indicate a text and drivingsituation generated from a numeric score. The score and flag can bepassed to a requesting application such as keyboard module.

In step 420, a machine learning module, can be updated with recentactivity in order to improve the accuracy of the generated score. Forexample, if a score is confirmed to indicate that a distracted drivercondition exists, this information can be used to improve the machinelearning module. Similarly, if a score is shown to be incorrect, thisinformation can also improve the machine learning module.

EXAMPLES

Some example data is set forth below for a “Person 1” in a safe textingenvironment, user average safe texting data, average user texting anddriving data, and example current data for a person.

EXAMPLES

Example Control Example Control Example Control Data: Data: Data: Person1 Safe User Average Safe Average User Example Current Texting TextingTexting and Driving Data: Factor Characteristics CharacteristicsCharacteristics Person 1 Gyro Sensor Angle = [−44, −8, −10] Angle =[−51, −12, −14] Angle = [−22, −22, −12] Angle = [−32, −15, −16] Averagevibration = Average vibration = Average vibration = Average vibration =0.023 g 0.026 g 0.078 g 0.072 g Average Average Average acceleration =Average acceleration = acceleration = 0.003 m/s2 acceleration = 0.002m/s2 0.012 m/s2 0.013 m/s2 Front Camera Average eye gaze Average eyegaze Average eye gaze Average eye gaze alternating between alternatingbetween alternating between alternating between screen and far screenand far screen and far screen and far object object (5 seconds) = object(5 seconds) = object (5 seconds) = (5 seconds) = 2.6 0.23 0.32 2.1 BackCamera Average light level Average light level Average light level (5Average light level (5 (5 seconds) that (5 seconds) that seconds) thatcamera seconds) that camera camera detects = camera detects = detects =314 lumens detects = 288 lumens 482 lumens 421 lumens Average proximityof Average proximity of Average proximity Average proximity closestobject = 0.3 closest object = 0.3 of closest object = of closest object= meters. meters. 2.3 meters. 2.1 meters. Proximity Average occupancyAverage occupancy Average occupancy Average occupancy (5 Sensor (5seconds) = (5 seconds) = (5 seconds) = seconds) = 0.2/second 0.2/second0.001/second 0.002/second Statistical Average texting in Average textingin Current hour text Current hour text driving info this hour, in thethis hour, in the amount = 1.4 texts. amount = 1.5 texts. same week day= same week day = 0.2 texts. 0.3 texts. Touch & Type Two hands texting =Two hands texting = Two hands texting = Two hands texting = trueDeletionrate = trueDeletion rate = falseDeletion rate = falseDeletion rate = 0.1per wordSurface 0.2 per wordSurface 0.3 per wordSurface 0.4 perwordSurface area (average 5 area (average 5 area (average 5 area(average 5 seconds) = 88 mm2 seconds) = 71 mm2 seconds) = 112 mm2seconds) = 137 mm2 Touch time Touch time Touch time Touch time (averageper key) = (average per key) = (average per key) = (average per key) =112 ms 85 ms 134 ms 152 ms Light Sensor Average light level Averagelight level Average light level (5 Average light level (5 (5 seconds)that (5 seconds) that seconds) that light seconds) that light lightsensor detects = light sensor detects = sensor detects = 245 sensordetects = 212 314 lumens 344 lumens lumens lumens Bluetooth ConnectedConnected Connected Bluetooth = 1 Connected Bluetooth = 1 Bluetooth =none Bluetooth = none Other phones Available Bluetooth = 3 AvailableBluetooth = 2 Available Bluetooth = 1 Available Bluetooth = 2 nearbyCommunicating Car sensor = Not Car sensor = Not Car sensor = Not Carsensor = Driver with the car avaialable avaialable avaialable onlyVoice/speech? Conversating in last Conversating in last Conversating inlast 5 Conversating in last 5 100 minutes = 0.2 100 minutes = 0.4minutes = 0 minutes = 0 conversations conversations conversationsconversations Face and Face attributes Not applicable Common faceDifferent than safe- behavior baseline for person 1 characteristics oftext texting attributes of analytics and drivers. Person 1, i.e. eyebrows raised significantly more times than safe texting time.

The above data is shown in FIGS. 5A-5D with the addition of a valuecolumn (Value 1, Value 2, Value) indicating an example weight for theparticular sensor data. Example weighting values in include 1, 2, and 3,where 3 is more heavily weighted than a 1. In example embodiments,sensor data with a Value of 3 are more heavily weight than 2 or 3. Forexample, gyro sensor data has a weighing value of 3 for the highestweighting value in the example embodiment. It is understood that anypractical weighting technique can be used to meet the needs of aparticular application.

A score column (Score 1, Score 2, Score 3) is also added in FIGS. 5A-5D.The score provides a relative value for the sensor data indicative ofthe likelihood of a distracted driving condition. That is, each scorecan be high or low depending upon the likelihood of a drive and textcondition. Distracted driving scores of 55, 20, and 16 derived from thescore and weighting values are shown in FIG. 5D. As will be appreciated,the score of 55 is indicative of a texting and driving situation.

In embodiments, distraction detection module 126 uses available sensordata, statistics of current and aggregated users, and/or criteria rulesto generate a distracted driving score. In one embodiment, the higherthe score, the more likely there is a text and driving situation. Thescore calculation method may be updated by machine learning for higheraccuracy and efficiency, and thus reduce false positives and negatives.Each criteria element, such as data from gyro-accelerator sensor 130,may have a specific weight in the score calculation. This weight may beupdated by machine learning. Each criteria element has a specific valueindicating the likeliness of distracted driving which is achieved bycomparing the current sensor data with reference values like safe andunsafe condition baselines of the same user, and safe and unsafecondition baselines of all users aggregated.

For illustration purposes, below is an example showing how a scoreindicating the likelihood of a distracted driving situation may becomputed, using two criteria elements for simplicity:

distracted driving score=sum(each criteria element's value*each criteriaelement's weight)=sum(front camera value*weight; other devices nearbyscore*weight)

The front camera value in this example is 3 because user's eyes and headmovements indicate he/she is looking at the screen and a far aheadobject, alternating more frequently than the safe baseline of the sameuser and all users aggregated. This indicates a likely drivingsituation. The other devices nearby score is 3 because there are nonearby devices detected via Bluetooth or similar protocols, indicatingthe user is likely alone in this moving environment and driving thevehicle.

=sum(3*3;3*1)

=12

Each value is normalized with a desired weight, in this examplemultiplied. It is understood that weights can be implemented in anysuitable way. In example embodiments, weighted values are summed togenerate a total score. In this example, 12 is a sufficiently high scoreso as to be indicative of texting and driving.

One can take another example using the similar logic:

distracted driving score=sum(front camera value*weight; other devicesnearby score*weight)=sum(1*3;0*1)=3

This time the front camera value is 1 because user's eye and headmovement is only slightly different than safe operation baseline of thesame users and aggregated users. The other devices nearby score is 0(zero) because there are 6 other devices nearby that broadcast Bluetoothsignals. This user is likely in a public transportation and not textingand driving. The score in this case is 3 which is significantly lowerthan 12 in the previous example. If this user in the publictransportation was the bus driver, the score would have been 9 due tohigh eye and head movements score, which as well would indicate a likelytext and driving situation.

In cases where only some of the sensor data is available, thedistraction detection module can dynamically adjust the weight and scorecalculations to accommodate the missing sensor data in order to achieveaccuracy.

A further example is shown in Table 1 below. Columns G and J are currentdata points that the distraction detector compares against data incolumns C, D, and E which are statistical reference points (controlgroup). Column G has the current (angle, vibration, and acceleration)values and they are different than the values in columns C and D(non-texting characteristics), and closer to column E (averagetext-and-drive characteristics). In general, during text-and-drivesituations, the driver holds the phone in a different angle toaccommodate steering wheel obstruction and positions the device as thatthe device is not too far from the road view. Also, due to multitasking,driver changes the angle of the phone several times causing high averageacceleration. As a result, a relatively high score indicative of drivingand texting is generated.

If we put the same person in the passenger seat this time, the values incolumn J are not expected to differ too much from column C and D, as aresult it scores low.

TABLE 1 C D E J Example Example Example G Example Control Data: ControlData: Control Data: Example Current Data: Person 1 Consumers AverageConsumer Current Data: Person 1 Safe Average Safe Texting and F Person 1H I (assumption = K L A B Texting Texting Driving Weight (assumption =Value Score in the passenger Value Score Factor Logic CharacteristicsCharacteristics Characteristics in Score driving a car) 1 1 seat) 2 2Gyro Angle = Angle = Angle = 3 Angle = 3 9 Angle = 1 3 Sensor [−44, −8,−10] [−51, −12, −14] [−22, −22, −12] [−32, −15, −16] [−41, −11, −11]Average Average Average Average Average vibration = vibration =vibration = vibration = vibration = 0.023 g 0.026 g 0.078 g 0.072 g0.044 g Average Average Average Average Average acceleration =acceleration = acceleration = acceleration = acceleration = 0.003 m/s20.002 m/s2 0.012 m/s2 0.013 m/s2 0.007 m/s2

In embodiments, the weight of each data point and score calculationlogic may be updated by a machine learning module. In embodiments, theweight of gyro sensor (gyro-accelerometer) is high because most usershold their devices differently when they text and drive than in astationary environment, such as texting on a couch. However, machinelearning may update the weight to achieve higher accuracies. Inembodiments, the weight of voice/speech is low because it may have higherror rates as the device may inaccurately determine that there are twopeople talking in the vehicle, which would decrease the possibility oftext and drive, however, such audio may be generated by two peopletalking on the radio, for example.

An example weighting and priority configuration is set forth below:

Weight Sensor data in Score Gyro Sensor 3 Front Camera 3 Touch & Type 3Statistical driving info 2 Bluetooth 2 Face and behavior analytics 2Back Camera 1 Proximity Sensor 1 Light Sensor 1 Other phones nearby 1Voice/speech? 1 Communicating with the car 0

FIG. 6 shows an exemplary computer 600 that can perform at least part ofthe processing described herein. The computer 600 includes a processor602, a volatile memory 604, a non-volatile memory 606 (e.g., hard disk),an output device 607 and a graphical user interface (GUI) 608 (e.g., amouse, a keyboard, a display, for example). The non-volatile memory 606stores computer instructions 612, an operating system 616 and data 618.In one example, the computer instructions 612 are executed by theprocessor 602 out of volatile memory 604. In one embodiment, an article620 comprises non-transitory computer-readable instructions.

Processing may be implemented in hardware, software, or a combination ofthe two. Processing may be implemented in computer programs executed onprogrammable computers/machines that each includes a processor, astorage medium or other article of manufacture that is readable by theprocessor (including volatile and non-volatile memory and/or storageelements), at least one input device, and one or more output devices.Program code may be applied to data entered using an input device toperform processing and to generate output information.

The system can perform processing, at least in part, via a computerprogram product, (e.g., in a machine-readable storage device), forexecution by, or to control the operation of, data processing apparatus(e.g., a programmable processor, a computer, or multiple computers).Each such program may be implemented in a high level procedural orobject-oriented programming language to communicate with a computersystem. However, the programs may be implemented in assembly or machinelanguage. The language may be a compiled or an interpreted language andit may be deployed in any form, including as a stand-alone program or asa module, component, subroutine, or other unit suitable for use in acomputing environment. A computer program may be deployed to be executedon one computer or on multiple computers at one site or distributedacross multiple sites and interconnected by a communication network. Acomputer program may be stored on a storage medium or device (e.g.,CD-ROM, hard disk, or magnetic diskette) that is readable by a generalor special purpose programmable computer for configuring and operatingthe computer when the storage medium or device is read by the computer.Processing may also be implemented as a machine-readable storage medium,configured with a computer program, where upon execution, instructionsin the computer program cause the computer to operate. The scorecalculation may be done locally on the handheld device or by a remotecomputer such as cloud computing.

Processing may be performed by one or more programmable processorsexecuting one or more computer programs to perform the functions of thesystem. All or part of the system may be implemented as, special purposelogic circuitry (e.g., an FPGA (field programmable gate array) and/or anASIC (application-specific integrated circuit)).

Having described exemplary embodiments of the invention, it will nowbecome apparent to one of ordinary skill in the art that otherembodiments incorporating their concepts may also be used. Theembodiments contained herein should not be limited to disclosedembodiments but rather should be limited only by the spirit and scope ofthe appended claims. All publications and references cited herein areexpressly incorporated herein by reference in their entirety.

What is claimed is:
 1. A method, comprising: receiving sensorinformation including GPS data to determine whether a computing deviceunder control of a user is moving relative to Earth surface; receivingsensor information including vibration levels of the device; receivingsensor information including angle orientation information for thedevice; receiving sensor information including first camera informationfor the device to detect at least one of user head and eye movement;processing the sensor information to determine a score corresponding toa likelihood of a distracted user condition; and communicating with akeyboard module of the device to modify at least one setting foroperation of a keyboard controlled by the keyboard module, thecommunicating including providing the keyboard module with at least oneof (i) an indication of a distracted user condition (ii) the scorecorresponding to the likelihood of a distracted user condition.
 2. Themethod according to claim 1, further including receiving sensorinformation including data from a first camera of the device andprocessing eye movement of the user.
 3. The method according to claim 1,further including receiving sensor information including touch and typeinformation for a user typing on the keyboard of the device.
 4. Themethod according to claim 3, further including processing the touch andtype information to determine whether the user is one-hand typing ortwo-hand typing on the keyboard.
 5. The method according to claim 3,further including processing the touch and type information for errorrate comparison.
 6. The method according to claim 3, further includingprocessing the touch and type information for finger surface area on thekeyboard.
 7. The method according to claim 3, further includingprocessing the touch and type information for speed of typingcomparison.
 8. The method according to claim 1, further includingprocessing historical driving information for the user including time ofday historical driving information.
 9. The method according to claim 1,further including receiving sensor information including local wirelessconnection information.
 10. The method according to claim 1, furtherincluding receiving sensor information including second camerainformation from the device that includes light level.
 11. The methodaccording to claim 1, further including receiving sensor informationincluding data from a proximity sensor of the device.
 12. The methodaccording to claim 1, further including receiving sensor informationincluding data from a light sensor of the device.
 13. The methodaccording to claim 1, further including receiving sensor informationincluding data for a number of other nearby devices.
 14. The methodaccording to claim 1, further including receiving sensor informationincluding acoustic information detected by the device to determine anumber of persons in a vehicle.
 15. The method according to claim 1,further including receiving sensor information including face andbehavior information of the user.
 16. The method according to claim 1,further including generating a signal for an audible alert correspondingto the score corresponding to a likelihood of a distracted usercondition being above a selected threshold.
 17. The method according toclaim 1, wherein modifying at least one setting for operation of akeyboard controlled by the keyboard module including disabling thekeyboard.
 18. An article, comprising: a non-transitory computer-readablemedium having stored instructions that enable a machine to perform:receiving sensor information including GPS data to determine whether acomputing device under control of a user is moving relative to Earthsurface; receiving sensor information including vibration levels of thedevice; receiving sensor information including angle orientationinformation for the device; receiving sensor information including firstcamera information for the device to detect at least one of user headand eye movement; processing the sensor information to determine a scorecorresponding to a likelihood of a distracted user condition; andcommunicating with a keyboard module of the device to modify at leastone setting for operation of a keyboard controlled by the keyboardmodule, the communicating including providing the keyboard module withat least one of (i) an indication of a distracted user condition and(ii) the score corresponding to the likelihood of a distracted usercondition.
 19. A device, comprising: a processor and a memory, theprocessor being configured to: receiving sensor information includingGPS data to determine whether the device is moving relative to Earthsurface; receiving sensor information including vibration levels of thedevice; receiving sensor information including angle orientationinformation for the device; receiving sensor information including firstcamera information for the device to detect at least one of user headand eye movement; generating a score corresponding to a likelihood of adistracted user condition based on the received sensor information; andin response to detecting that the score corresponding to the likelihoodof a distracted user condition exceeds a threshold, providing a keyboardmodule with at least one of (i) an indication of a distracted usercondition and (ii) the score corresponding to the likelihood of adistracted user condition, wherein the keyboard module is configured tomodify at least one setting for operation of a keyboard controlled bythe keyboard module based on the score corresponding to the likelihoodof a distracted user condition.